The present invention relates to a memory management method, an image coding method, an image decoding method, image display method, a memory management apparatus, an image coding apparatus, an image decoding apparatus, an image display apparatus, a memory management program storage medium, an image coding program storage medium, an image decoding program storage medium and an image display program storage medium.
More particularly, the present invention relates to a memory management method which can reduce a memory capacity required for coding, decoding or displaying plural image sequences simultaneously, a method for coding images, decoding images and displaying images utilizing this memory, an apparatus corresponding to these memory management methods and method for coding images, decoding images and displaying images, and a storage medium which contains a program for implementing the respective methods by software.
In recent years, we have greeted the age of multimedia in which audio, video and other pixel values are integrally handled, and the conventional information media (i.e., means for transmitting information to people), such as newspapers, magazines, televisions, radios, and telephones, have been adopted as the subjects of multimedia.
Generally, xe2x80x9cmultimediaxe2x80x9d means to represent, not only characters, but also diagrams, speech, and especially images simultaneously in relation with each other. In order to handle the conventional information media as the subjects of multimedia, it is necessary to transform the information into a digital format.
When the quantity of data possessed by each information medium described above is estimated as the quantity of digital data, in cases of characters, the data quantity for each character is 12 byte. However, in cases of speech, the data quantity is 64 kbits per second (quality for telecommunication) and, in cases of moving pictures, it is more than 100 Mbits per second (quality for current television broadcasting). So, as for the information media described above, it is not practical to handle such massive data as it is in the digital format. For example, although visual telephones have already been put to practical use by ISDN (Integrated Services Digital Network) having a transmission rate of 64 kbps-1.5 Mbps, it is impossible to transmit an image of a television camera as it is by the ISDN.
So, data compression technologies are demanded. In cases of visual telephones, the moving picture compression technologies standardized as H.261 and H.263 by ITU-T (International Telecommunication Union-Telecommunication Sector) are employed. Further, according to the data compression technology based on MPEG1, it is possible to record image data, together with audio data, in an ordinary music CD (compact disk).
MPEG (Moving Picture Experts Group) is an international standard of data compression for a pixel value of a moving picture. In MPEG1, a pixel value of a moving picture is compressed to 1.5 Mbps, i.e., data of a television signal is compressed to about {fraction (1/100)}. Since the transmission rate to which MPEG1 is directed is limited to about 1.5 Mbps, MPEG2 is standardized to meet the demand for higher image quality. In MPEG2, a pixel value of a moving picture is compressed to 2xcx9c15 Mbps.
Under the existing circumstances, standardization of MPEG4 is now proceeded by the working group for standardization of MPEG1 and MPEG2 (ISO/IEC JTC1/SC29/WG11). MPEG4 enables coding and processing in units of objects included in an image, and thereby realizes new functions required in the age of multimedia. MPEG4 had originally aimed at standardization of a coding method at a low bit rate, but the aim of standardization has now extended to a more versatile coding process including coding of an interlace image and coding at a high bit rate.
One of the characteristics of MPEG4 is a mechanism for coding and transmitting plural image sequences simultaneously. According to this mechanism, one image scene can be composed of plural images. For example, different image sequences can be used as foreground and background, whereby their frame frequencies, image qualities or bit rates are changed individually. Accordingly, plural image sequences are arranged in the horizontal or vertical direction like a multi-screen on a television receiver or the like, whereby the user can extract or zoom only desired image sequences.
FIG. 15 is a diagram showing a typical example where MPEG coding is performed to an image sequence. This figure shows an example of a frame structure according to MPEG1 or MPEG2. In MPEG4, this corresponds to a reference structure of a VOP of a certain object.
In MPEG coding, there are a frame (I frame) which is coded only by intra-frame operation, a frame (P frame) which is subjected to inter-frame prediction coding performed utilizing the correlation between frames, and a frame (B frame) which is subjected to bidirectional prediction coding performed on the basis of the past frame and the future frame.
FIG. 15 shows the case where MPEG coding is performed to contiguous eight frames FR1, FR2, . . . , FR8. The frames FR1 and FR5 are I frames. Other frames, i.e., FR2, FR3, FR4, FR6, FR7 and FR8, are P frames.
The I frame can be decoded by itself. However, the P frame cannot be coded or decoded when a frame at the previous time, which is to be referred to, has not been coded or decoded correctly. Therefore, when a transmission error of a bit stream or failure at decoding time occurs, correct image data cannot be obtained until the next I frame is input.
In coding or decoding the P frame, a reference frame is required. Accordingly, the reference frame and one frame for recording data in the process of coding or decoding the present frame, i.e., two frames of the memory, are required for encoding or decoding of one image sequence (object). Similarly, 2n frames of memory area are required for encoding or decoding of n image sequences. Accordingly, in the prior art, the memory address space is divided into 2n pieces, each of the divided areas (banks) is used as one frame of the memory, and the image data is stored therein.
FIG. 16 shows a conventional way of dividing the memory address. In this figure, FM1a, FM1b, FM2a, FM2b, FM3a and FM3b denote a first frame area of a first image sequence, a second frame area of the first image sequence, a first frame area of a second image sequence, a second frame area of the second image sequence, a first frame area of a third image sequence and a second frame sequence of the third image sequence, respectively.
In addition, AD1a, AD1b, AD2a, AD2b, AD3a and AD3b denote a first address start location of the first image sequence, a second address start location of the first image sequence, a first address start location of the second image sequence, a second address start location of the second image sequence, a first address start location of the third image sequence and a second address start location of the third image sequence, respectively. AD1a to AD1b-1 corresponds to a 1a-th memory bank. AD1b to AD2a-1 corresponds to a 1b-th memory bank. AD2a to AD2b-1 corresponds to a 2a-th memory bank. AD2b to AD3a-1 corresponds to a 2b-th memory bank. AS3a to AD3b-1 corresponds to a 3a-th memory bank.
SZ1a, SZ1b, SZ2a, SZ2b, SZ3a and SZ3b denote a first frame size of the first image sequence, a second frame size of the first image sequence, a first frame size of the second image sequence, a second frame size of the second image sequence, a first frame size of the third image sequence, and a second frame size of the third image sequence, respectively.
Here, the way of storing image data of each image sequence is described taking the case of decoding the frames FR1, FR2, . . . , FR8 in FIG. 15 as an example.
Assume that these frames FR1, FR2, . . . , FR8 constitute one image sequence (assuming the first image sequence). This image sequence corresponds to one object in MPEG4 and one GOP in MPEG2.
Initially, the frame FR1 as the first I frame in the first image sequence is decoded and a decoded image data is stored in the frame area FM1a. 
Next, the frame FR2 is decoded with reference to the decoded image data of the frame FR1 which has been stored in the frame area FM1a, and the result of the decoding is stored in the frame area FM1b. 
Then, the frame FR3 is decoded with reference to the decoded image data of the frame FR2 which has been stored in the frame area FM1b, and the result of the decoding is stored in the frame area FM1a. That is because the image data of the frame FR1 is no longer required for decoding the frame FR3 and therefore the data in the frame area FM1a can be overwritten by the decoded image data of the frame FR3.
Further, the frame FR4 is decoded with reference to the decoded image data of the frame FR3 which has been stored in the frame area FM1a and the result of the decoding is stored in the frame area FM1b. 
Hereinafter, the frames FR5 to FR8 of the first image sequence are similarly decoded with the use of the two frame areas FM1a and FM1b alternately.
The frames of the second and third image sequence are decoded similarly. To be specific, the frames of the second image sequence are decoded with the use of the two frame areas FM2a and FM2b alternately and the frames of the third image sequence are decoded with the use of the two frame areas FM3a and FM3b alternately, respectively.
Also, when the coding such as inter-frame prediction coding or the display of images is performed, the coding or image display is performed similarly by alternately using two frame areas which are adjacent to each other in the memory.
While I frame or P frame is referenced to decode P frame or B frame, B frame is not referenced to decode another B frames or P frame. Therefore, B frame is not stored in the memory.
If B frame exists between the frames FR2 and FR3, this B frame is decoded with referring to the frame FR2 which has been stored in the frame area FM1b and the frame FR3 which has been stored in the frame area FM1a. 
In each of the frame areas FM1a, FM1b, FM2a, FM2b, FM3a and FM3b, data are stored from a lower address to a higher address in each area (bank) of the image memory, taking each of the address start locations AD1a, AD1b, AD2a, AD2b, AD3a and AD3b as the origin, as shown by the arrow in FIG. 16. Therefore, even when each of the frame sizes SZ1a, SZ1b, SZ2a, SZ2b, SZ3a and SZ3b is increased, the image data required for coding or decoding of the image sequence are not destroyed by image data of another image sequence, unless the increased frame size exceeds the size of each bank of the divided memory. Thereby, when the size of the memory area required for coding or decoding of each image sequence is fixed, the structure in which the memory address is divided as shown in FIG. 16 has no problem.
FIG. 17 is a block diagram illustrating a prior art image coding apparatus.
As shown in this figure, a motion estimation/motion compensation (MEMC) unit U1 reads out a reference frame FMout from a memory unit FM for an image signal Vin, performs motion estimation/motion compensation with reference to the reference frame, and outputs a motion vector MV and a motion compensated image Ref. A subtracter U2 calculates the difference between the image signal Vin and the motion compensation frame Ref, and outputs the difference to a discrete cosine transformation (DCT) unit U3. The discrete cosine transformation (DCT) unit U3 performs DCT transformation in block units. A quantization (Q) unit U4 quantizes the result of the transformation. A variable-length coding (VLC) unit U5 subjects the quantized value to variable-length coding and outputs a coded stream Vbin.
On the other hand, the quantized value output by the quantization (Q) unit U4 is subjected to inverse quantization by an inverse quantization (IQ) unit U6. Then, inverse DCT transformation in block units is performed by an inverse discrete cosine transformation (IDCT) unit U7. Thereafter, the output of the inverse discrete cosine transformation (IDCT) unit U7 is added to the motion compensation frame Ref and a frame FMin to be recorded in the memory unit FM is obtained. In coding I frame, the reference frame is not required. In this case, all pixel values of the motion compensation frame Ref are handled as xe2x80x9c0xe2x80x9d.
The sequence identification number ObjID of the image and the size ObjSZ of each frame are input to the memory unit FM. The sequence identification number ObjID of the image is used to specify the sequence of the image. The size ObjSZ of each frame is used to designate the size of the memory area required for recording of the frame FMin to be recorded. For example, in FIG. 16, ObjID and ObjSZ of FM1a are represented as ObjID=1 and ObjSZ=SZ1a. ObjID and ObjSZ of FM2b are represented as ObjID=2 and ObjSZ=SZ2b. 
FIG. 18 is a block diagram illustrating a prior art image decoding apparatus. In this figure, a variable-length decoding (VLD) unit U11 subjects a coded stream Vbin to variable-length decoding. An inverse quantization (IQ) unit U6 inversely quantizes the result of the variable-length decoding. An inverse discrete cosine transformation (IDCT) unit U7 subjects the result of the inverse quantization to inverse DCT transformation in block units.
A memory unit FM outputs a reference frame FMout corresponding to the sequence identification number ObjID of the image. A motion compensation (MC) unit U12 performs motion compensation corresponding to a motion vector MV to generate a motion compensation frame Ref. An adder U8 adds the motion compensation frame Ref and the output of the inverse discrete cosine transformation (IDCT) unit U7 to obtain a decoded frame signal Vout. This frame signal Vout is recorded in the memory unit FM as a record frame FMin composed of image data having the size ObjSZ in a memory area designated by the sequence identification number ObjID of the image. The reference frame is not required for decoding of I frame. In this case, all pixels in the motion compensation frame Ref are handled as xe2x80x9c0xe2x80x9d.
FIG. 19 is a block diagram illustrating a prior art image display apparatus. In this figure, an object selector U20 successively instructs the memory unit FM of the number of an image sequence to be displayed as ObjID in accordance with information ObjSe1 of the image sequence to be displayed which is instructed by the user. An image output FMout output by the memory unit FM in accordance with this instruction is input to a composer U23. Then, this image output FMout is either composed with another image sequence output by the memory unit FM and displayed, or composed with an already determined background frame BG selected by a switch U21, output to a switch U24, and a composed image is displayed. At a time when all image sequences to be displayed have been read from the memory unit FM, the object selector U20 notifies the switch U24 with an end signal ObjEnd that reading of the image sequences to be displayed is ended. Thereby, the switch U24 outputs a composite image to a display device U25 at a timing of a display synchronous signal Display.
In the usual construction, the image decoding apparatus of FIG. 18 and the image display apparatus of FIG. 19 are constructed as an integrated apparatus in many cases. In these cases, the image decoding apparatus of FIG. 18 and the image display apparatus of FIG. 19 often use the memory unit FM commonly, thereby reducing the memory to half as compared with the case where these apparatus are constructed as individual apparatus.
FIG. 20 is a block diagram illustrating a prior art memory unit FM. When data are to be recorded in the memory unit FM, a bank selector M1 specifies a memory bank among divided memory banks corresponding to a sequence identification number ObjID of an image. An address generator M2 generates an address corresponding to a memory area having a size designated by the size ObjSZ of the image. A memory M4 contains a frame signal FMin in the address space.
Similarly, when data are to be read from the memory unit FM, the bank selector M1 specifies a memory area among the divided memory banks corresponding to the sequence identification number ObjID of an image. The address generator M2 generates an address corresponding to a memory area having the size designated by the size ObjSZ of the image. A frame signal FMout is read from the address space of the memory M4.
Accordingly, when the decoding is performed, the reading/writing is performed with alternately utilizing respective frame areas in two memory banks which are adjacent to each other, by using the access function in memory bank units included in the memory unit FM. Thereby, the image sequence corresponding to each object can be decoded.
The memory management method corresponding to the prior art image sequence is constructed as described above. The management of the memory is performed so as to reserve the bank corresponding to two frames of the memory in total which are required for coding or decoding of one image sequence in the memory.
However, in the prior art memory address management method as shown in FIG. 16, when the frame signal which is temporally divided exceeds the maximum value of each memory, this frame signal cannot be recorded. To be specific, the frame does not greatly vary in size in the middle of the image sequence in MPEG1 or MPEG2, while the size of the frame is allowed to vary with time in MPEG4. When the size of the frame signal to be recorded exceeds the size of the divided memory bank, frame data which are required for coding, decoding or display may be erased. When this problem is to be avoided, a sufficient memory bank is reserved so that each of the divided memory banks has the maximum value of the permissible frame size. However, in such a case, an enormous memory capacity is required and accordingly the power consumption of the device involving a memory is increased.
Further, in the prior art, the variation with time in the size of the frame signal is not considered. Therefore, no measures are taken to deal with cases where frame data which are required for decoding or display are erased in the decoding apparatus or display apparatus.
It is an object of the present invention to provide a memory management method, a memory management apparatus and a memory management program storage medium, which can increase the efficiency of the memory bank and greatly reduce the risk of erasure of frame data which are required for coding, decoding or display.
It is another object of the present invention to realize an image decoding method, an image decoding apparatus, an image decoding program storage medium, an image display method, an image display apparatus and an image display program storage medium, which reduce deterioration of image quality when frame data required for decoding or display are erased, an image decoding method, an image decoding apparatus and an image decoding program storage medium, which can reduce an operation amount, and an image coding method, an image coding apparatus and image coding program storage medium, which perform coding corresponding to the decoding.
Other objects and advantages of the present invention will become apparent from the detailed description and specific embodiments described are provided only for illustration since various additions and modifications within the spirit and scope of the invention will be apparent to those of skill in the art from the detailed description.
A memory management method according to a 1st aspect of the present invention for simultaneously recording image data of first to n-th image sequences (n is an integer which is equal to or larger than 2) in a memory comprises steps of: dividing an area of the memory into first to n/2-th memory address spaces by allocating addresses from ADstart[i] to ADend[i] (i is an integer which satisfies 1xe2x89xa6ixe2x89xa6n/2) to a i-th memory address space, respectively; using an area from one of ends of ADstart[i] and ADend[i] of the i-th memory address space toward one of a higher and lower memory addresses in the i-th memory address space, as an area in which image data of a k-th image sequence (k=ixc3x972xe2x88x921) are recorded; and using an area from the other of the ends of ADstart[i] and ADend[i] toward the other of the lower and higher memory addresses in the i-th memory address space, as an area in which image data of a (k+1)-th image sequence are recorded. Therefore, the memory management method of high practical value, which can increase the efficiency of the memory area, and greatly reduce the risk of erasure of frame data which are required for the coding, decoding or display, can be realized.
According to a 2nd aspect of the present invention, the memory management method of the 1st aspect comprises steps of: monitoring when the image data of the k-th image sequence and the image data of the (k+1)-th image sequence are recorded together in the i-th memory address space, whether one of the image data are overwritten by the other of the image data; and notifying outside when image data are read from the memory, whether image data of an image sequence to be read out have been destroyed by the overwriting. Therefore, the memory management method of high practical value, which can increase the efficiency of the memory area, greatly reduce the risk of erasure of frame data which are required for the coding, decoding or display, and notify the outside whether the image of the accessed image sequence is destroyed or not, can be realized.
According to a 3rd aspect of the present invention, the memory management method of the 1st aspect comprises steps of: monitoring when the image data of the k-th image sequence and the image data of the (k+1)-th image sequence are recorded together in the i-th memory address space, whether one of the image data are overwritten by the other of the image data; and referring to priority information of the image sequences which is input from outside when image data are destroyed by the overwriting, and performing the overwriting so as to destroy image data having a lower priority and protecting image data having a higher priority. Therefore, the memory management method of high practical value, which can increase the efficiency of the memory area, greatly reduce the risk of erasure of frame data which are required for the coding, decoding or display, and reduce the deterioration of image quality even when the frame data are erased, can be realized.
An image coding method according to a 4th aspect of the present invention for simultaneously coding image data of first to n-th image sequences (n is an integer which is equal to or larger than 2) comprises a step of: recording predictive image data in the memory which is managed according to the memory management method of the 1st aspect when inter-frame prediction coding is performed. Therefore, the image coding method of high practical value, which can increase the efficiency of the memory area and greatly reduce the risk of erasure of frame data which are required for the coding, can be realized.
An image decoding method according to a 5th aspect of the present invention for simultaneously decoding image data of first to n-th image sequences (n is an integer which is equal to or larger than 2) comprises a step of: recording predictive image data in the memory which is managed according to the memory management method of the 1st aspect when inter-frame prediction decoding is performed. Therefore, the image decoding method of high practical value, which can increase the efficiency of the memory area and greatly reduce the risk of erasure of frame data which are required for the decoding, can be realized.
An image decoding method according to a 6th aspect of the present invention for simultaneously decoding image data of first to n-th image sequences (n is an integer which is equal to or larger than 2) comprises steps of: recording predictive image data in the memory which is managed by the memory management method of the 2nd aspect when inter-frame prediction decoding is performed; and interrupting a decoding process when image data in the memory which are required for the decoding of the image sequence are destroyed, until decoding can be performed without referring to the image data in the memory. Therefore, the image decoding method of high practical value, which can increase the efficiency of the memory area, greatly reduce the risk of erasure of frame data which are required for the decoding, and reduce the deterioration of image quality when the frame data which are required for the decoding are erased, can be realized.
An image decoding method according to a 7th aspect of the present invention for simultaneously decoding image data of first to n-th image sequences (n is an integer which is equal to or larger than 2) comprises steps of: when inter-frame prediction decoding is performed, recording predictive image data in the memory which is managed according to the memory management method of the 3rd aspect so that an important image sequence has a higher priority; and making image data which are required for decoding of the important image sequence hard to be destroyed. Therefore, the image decoding method of high practical value, which can increase the efficiency of the memory area, greatly reduce the risk of erasure of frame data which are required for the decoding, and reduce the deterioration of image quality even when the frame data required for the decoding are erased, can be realized.
An image display method according to an 8th aspect of the present invention for simultaneously displaying image data of first to n-th image sequences (n is an integer which is equal to or larger than 2) comprises a step of: recording image data to be displayed in a memory which is managed according to the memory management method of the 1st aspect. Therefore, the image display method of high practical value, which can increase the efficiency of the memory area and greatly reduce the risk of erasure of frame data which are required for the display, can be realized.
An image display method according to a 9th aspect of the present invention for simultaneously displaying image data of first to n-th image sequences (n is an integer which is equal to or larger than 2) comprises a step of: displaying when image data in a memory which are required for display of the image sequence are destroyed, image data at a latest time which are not destroyed in the image sequence, in place of the destroyed image data. Therefore, the image display method of high practical value, which can increase the efficiency of the memory area, greatly reduce the risk of erasure of frame data which are required for the display, and reduce the deterioration of image quality when the frame data required for the display are erased, can be realized.
A memory management apparatus according to a 10th aspect of the present invention for simultaneously recording image data of first to n-th image sequences (n is an integer which is equal to or larger than 2) in a memory comprises: a memory division unit for dividing an area in the memory into first to n/2-th memory address spaces by allocating addresses from ADstart[i] to ADend[i] (i is an integer which satisfies 1xe2x89xa6ixe2x89xa6n/2) as a i-th memory address space, respectively; and an address generation unit for generating an address for using an area from one of ends of ADstart[i] and ADend[i] of the i-th memory address space toward one of a higher and lower memory address in the i-th memory address, as an area in which image data of a k-th image sequence (k=ixc3x972xe2x88x921) are recorded, and using an area from the other of the ends of ADstart[i] and ADend[i] toward the other of the lower and higher memory address of the i-th memory address space as an area in which image data of a (k+1)-th image sequence are recorded. Therefore, the memory management apparatus of high practical value, which can increase the efficiency of the memory area and greatly reduce the risk of erasure of frame data which are required for the coding, decoding or display, can be realized.
According to an 11th aspect of the present invention, there is provided a memory management program storage medium which contains a memory management program for implementing a memory management method for simultaneously recording image data of first to n-th image sequences (n is an integer which is equal to or larger than 2) in a memory, and this memory management program implementing the memory management method of: dividing an area in the memory into first to n/2 memory address spaces by allocating addresses from ADstart[i] to ADend[i] (i is an integer which satisfies 1xe2x89xa6ixe2x89xa6n/2) to a i-th memory address space, respectively; using an area from one of ends of ADstart[i] and ADend[i] of the i-th memory address space toward one of a higher and lower memory addresses in the i-th memory address space, as an area in which image data of a k-th image sequence (k=ixc3x972xe2x88x921) are recorded; and using an area from the other of ends of ADstart[i] and ADend[i] toward the other of the lower and higher memory addresses in the i-th memory address space, as an area in which image data of a (k+1)-th image sequence (k=ixc3x972xe2x88x921) are recorded. Therefore, the program storage medium containing a program which enables a computer to execute a memory management method of high practical value, which can increase the efficiency of the memory area and greatly reduce the risk of erasure of frame data which are required for the coding, decoding or display, can be provided.
In a memory management apparatus according to a 12th aspect of the present invention, when newly input image data are to be stored in a memory area which is a free space or contains i pieces of image data (i is an integer from 1 to 3), in a case where image data can be stored on at least one of a lowest side and a highest side of an address of the memory area, the newly input image data are stored on one of the lowest side and highest side of the address of the memory area; and in a case where image data cannot be stored in either the lowest side nor highest side of the address of the memory area, the newly input image data are stored in an intermediate part therebetween, whereby storage of the memory is managed, and the storage of the newly input image data in the intermediate part is either storing the newly input image data in an area which is adjacent to one of the image data which have been stored on the highest side and lowest side, or storing the newly input image data in an area which is adjacent to an address central location in the memory area. Therefore, the efficiency of the memory can be increased and the risk of the decoding incapability due to the exhaustion of the memory required for the decoding can be greatly reduced.
According to a 13th aspect of the present invention, in the memory management method of the 12th aspect, in a case where the image data which have been stored in the memory area and the newly input image data are all rectangular-shaped image data, when image data are to be stored in the intermediate part, memory areas for already stored image data are relocated so that the newly input image data are stored in an area which is adjacent to one of the image data which have been stored on the highest side and lowest side. Therefore, a larger memory amount can be reserved, whereby the efficiency of the memory can be increased and the risk of the decoding incapability due to the exhaustion of the memory required for the decoding can be greatly reduced.
According to a 14th aspect of the present invention, in the memory management method of the 13th aspect, the relocation of the memory areas for the already stored image data is performed only when there is not an enough memory area for the newly input image data. Therefore, the execution times of the data relocation process having a high risk of interruption of the decoding process can be reduced, whereby the efficiency of the memory can be increased as well as the risk of decoding incapability due to exhaustion of the memory required for the decoding can be greatly reduced. Accordingly, the memory management method of high practical value can be realized.
According to a 15th aspect of the present invention, in the memory management method of the 12th aspect, in a case where at least one among the image data stored in the memory area and the newly input image data is an arbitrary-shaped image data, when image data are to be stored in the intermediate part, memory areas for already stored image data are relocated so that the newly input data are stored in an area which is adjacent to an address central location in the memory area. Therefore, even when the size of an individual object varies in some degree, the memory area can be dynamically reserved, whereby the risk of the decoding incapability due to the exhaustion of the memory required for the decoding can be greatly reduced.
According to a 16th aspect of the present invention, in the memory management method of the 15th aspect, the relocation of the memory areas for the already stored image data is performed only when there is not an enough memory area for the newly input image data. Therefore, the execution times of the data relocation process having a high risk of interruption of the decoding can be reduced, whereby the efficiency of the memory can be increased as well as the risk of the decoding incapability due to the exhaustion of the memory required for the decoding can be greatly reduced. Accordingly, the memory management method of high practical value can be realized.
According to a 17th aspect of the present invention, in the memory management method of the 12th aspect, while the memory is being managed so as to store the newly input image data in an area which is adjacent to one of the image data which have been stored on the highest side and lowest side when image data are to be stored in the intermediate part, in a case where arbitrary-shaped image data are newly input, memory areas for already stored image data are relocated so that the newly input image data are stored in an area which is adjacent to the address central location in the memory area when the image data are to be stored in the intermediate part. Therefore, a larger memory amount can be reserved until an arbitrary-shaped object is input. In addition, after the arbitrary-shaped object is input, the sum of the memory amounts between two objects can be freely changed within the upper limit. Accordingly, the sufficient memory area for the decoding can be used with less relocation of data in the memory.
According to a 18th aspect of the present invention, in the memory management method of the 17th aspect, after the relocation of the memory areas resulting from input of the arbitrary-shaped image data, only when there is not an enough memory area for a newly input image data, the memory areas for the already stored image data are relocated so that the newly input image data are stored in an area which is adjacent to the address central location in the memory area when the image data are to be stored in the intermediate part. Therefore, the execution times of the data relocation process having a high risk of interruption of the decoding can be reduced, whereby the efficiency of the memory can be increased as well as the risk of the decoding incapability due to the exhaustion of the memory required for the decoding can be greatly reduced. Therefore, the memory management method of high practical value can be realized.